Techniques for translating content

ABSTRACT

Techniques, including systems and methods, for providing translated content are described and suggested herein. In an embodiment, translations for instances of content are received and reviewers provide votes regarding the translations&#39; quality. Votes received for translations are utilized in order to determine scores for the translations. Reviewers and translators may be scored as well. Scores for translations are used in various processes for providing appropriate content to users.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.12/881,605, filed Sep. 14, 2010, entitled “TECHNIQUES FOR TRANSLATINGCONTENT”, which is incorporated herein by reference in its entirety.

BACKGROUND

While current communications networks provide the ability to easilycommunicate to people all over the world, the content of a communicationis only effective when it is provided in a language that isunderstandable to those involved in the communication. A websiteproviding content in English, for instance, may not be very useful to anon-English speaking person. Generally in such situations, content inone language often must be translated to another language to beeffective. An entity wishing to expand operations of an electroniccommerce website into another country, for example, may need totranslate all or at least a portion of the website into the country'slocal language. Similarly, an entity wishing to provide services topopulations who do not communicate well in a primary language of ageographic area must translate much of the content that it distributes,whether electronically or otherwise. Complicating this issue is the factthat content is often generated at a high rate, especially when contentproviders give users the opportunity to provide their own content.

Partly due to the need to translate content and the amount of content tobe translated, automated translation tools have developed. However,because of the complexity of language, cultural differences in howpeople express themselves, various incongruent grammatical structures,and other factors, the quality of content translated by these automatedtranslation tools is not optimal. The meaning of a word, for instance,may change dramatically based on the context in which the word is used.The word “flash,” for example can refer to a burst of radiant energy, anillumination device of a camera, a brief moment in time, a type ofcomputer memory, or other things depending on how the word is used. Inanother language, a single word may not have the same multiple meanings.Automated translation tools have had limited success ascertaining thecorrect meaning of a word when translating. Thus, automated translationtools often provide content that is difficult to understand or simplynot elegantly composed. While manual translation by a qualifiedtranslator may address these issues, manual translation is often timeconsuming and prohibitively expensive for many uses when translationwould be beneficial.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an illustrative example of a Web page which may be producedin accordance with an embodiment.

FIG. 2 shows an illustrative example of a Web page on which contentsfrom the Web page shown in FIG. 1 has been translated in accordance withthe embodiment.

FIG. 3 shows an illustrative example of the Web page which includestranslated content of the Web page shown in FIG. 1 in accordance withanother embodiment.

FIG. 4 shows another illustrative example of a Web page, includingcontent translated from the content shown in FIG. 1 in accordance withan embodiment.

FIG. 5 shows an illustrative example of a Web page which may be used toallow for users to provide translations in accordance with anembodiment.

FIG. 6 shows an illustrative example of an embodiment in which variousembodiments can be implemented.

FIG. 7 shows a flowchart for an illustrative example of a process forproviding content in accordance with an embodiment.

FIG. 8 shows a flowchart for an illustrative example of a process forproviding translated content in accordance with an embodiment.

FIG. 9 shows a flowchart for an illustrative example of a process forgrading translator's translations and reviewers of translations inaccordance with an embodiment.

FIG. 10 shows a flowchart for an illustrative example of a process forscoring translation reviewers in accordance with an embodiment.

FIG. 11 illustrates another environment in which various embodiments canbe implemented.

FIG. 12 illustrates an environment having components which may beutilized in implementation of various embodiments, including embodimentsshown in FIGS. 6 and 11.

DETAILED DESCRIPTION

Techniques, including systems and methods, for translating content aredescribed and suggested herein. In an embodiment, various instances ofcontent are provided to users. Examples of content instances includedocuments, portions of text, audio files, video files, and, generally,any collection of information that encodes information expressed in alanguage. Providing content instances to users may be performed invarious ways, including, in an embodiment, sending content instances tousers over a communications network for display or other presentation onuser devices. Content instances, for example, may be encoded as part ofweb pages. In an embodiment, at least some users are provided anopportunity to translate content instances. As an example, some or allusers may be provided content with a selectable interface element thatcan be selected in order to allow the users to submit translations ofthe content. The submitted translations may be stored and used toprovide translations of the content to other users.

In an embodiment, submitted translations are rated by one or morereviewers. Reviewers may be people to whom translated content isprovided on a web page. Reviewers may submit electronic votes (a type ofreview) to express opinions about the quality of a particulartranslation. Reviewers may also be provided an opportunity to submittheir own translations and/or to correct translations that the reviewersbelieve are mistranslations. Votes received for translations may be usedto rate the translations. For example, if reviewers of a translationgenerally agree that the translation is a good translation, then thetranslation may be rated high or, generally favorably, such as by havingnine out of ten stars, or some other measure of the translation'squality. If reviewers generally agree that a translation is a poortranslation, then the translation may be rated low, or generallyunfavorably such as by being given a lower score or having a higherpercentage of unfavorable ratings. A translation's rating may be encodedin a translation score, which may be a numerical or other value thatindicates the translation's quality, as determined by reviewers thathave submitted votes for the translation. A translation's rating may beused to determine whether to provide the translation to users and/or todetermine in which contexts to provide the translations. For instance,if a user navigates to a web page about a particular item (e.g., productor service) offered for consumption (e.g., purchase, rent, lease, ordownload), if translated information related to the item is available ina language that the user understands and the translated information israted high or favorably enough, then the translated information may beprovided to the user. The information related to the item may be, forinstance, a user-generated review of the item submitted in a languageother than a language that the user understands.

Reviewers and translators may also be rated, in accordance with variousembodiments. Reviewers, for example, may be rated based at least in parton statistics regarding how their reviews agree with those submitted byother reviewers. The quality of a reviewer, based at least in part onhow the reviewer's reviews agree with other reviewers, may be encoded ina reviewer score, which may be a numerical or other value that indicatesthe quality of the reviewer. If others generally agree with a reviewerabout the quality of one or more translations, then the reviewer may berated high. If others generally disagree with a reviewer about thequality of one or more translations, then the reviewer may be rated low.Translators that have provided translations of content may be ratedbased at least in part on statistics regarding how their translationshave been rated. The quality of a translator, based at least in part onratings of the translator's translations, may be encoded in a translatorscore, which may be a numerical or other value that indicates thequality of the translator.

Reviewer scores, translator scores, and translation scores may be usedin calculations of each other. As an example, a translation score of apiece of content (or portion of content) may be based at least in parton the reviews of the translation by others. Reviews of the translationthat are associated with high-score reviewers may be more heavilyweighted than translation scores submitted by lower scoring reviewers.In this manner, a translator's translator score may depend at least inpart on scores of reviewers that have reviewed the translator'stranslations. As another example, a person's reviewer score may beaffected by their agreement with reviews submitted by high-scorereviewers (e.g., increases reviewer score) as opposed to reviewssubmitted by lower scoring reviewers (e.g., lowers reviewer score).

Translations provided according to various embodiments described andsuggested herein may be provided as a service to entities in need oftranslations. Entities may, for instance, request translations bycalling a web service operable to provide translations. An entity maysubmit content and one or more languages into which a translation of thecontent is desired. The submitted content may be provided by usersaccording to the various embodiments described and suggested herein.Translations may be then provided to the requestor.

FIG. 1 shows an illustrative example of a Web page 100 that includescontent in accordance with an embodiment. In this particular example,the Web page 100 is a Web page for providing user reviews of aparticular consumer product. For instance, as shown, the Web page 100includes a review 102 which was generated (composed) and submitted by auser. Proximate to the review in this example are various interfaceelements that allow users viewing the review 102 to provide input aboutthe review 102. For instance, the Web page 100 includes a yes button 104and a no button 106 that allow the user to provide input regardingwhether the user found the review 102 helpful. If, for example, the userviewing the review 102 found the review 102 helpful, then the user mayselect the yes button 104 and a device on which the Web page 100 isdisplayed may send a message indicating the user selection of the yesbutton 104 for recording. Similarly, the user may select a no button 106in order to indicate that he or she did not find the review helpful. Theuser's device may then send an appropriate message indicating the user'sselection of the no button 106.

In an embodiment, as different users select the yes button 104 or the nobutton 106, the users' selections are recorded and used to provideinformation about how users in general value the review. For instance,as shown in the Web page 100, a sentence appears saying that “Thisreview was helpful for 191 out of 206 shoppers.” Thus, in an embodimentusers may have selected either the yes button 104 or the no button 106,and 191 of those users may have selected the yes button 104. In thismanner, users can quickly ascertain, based at least in part on theopinions of other users, whether they want to read the review carefullyin making a purchasing or other consumption decision.

In the example Web page 100, content is provided in English. The Webpage 100 may, for instance, be provided as part of the electroniccommerce website that offers items for consumption to users in anEnglish speaking country, such as the United States, Canada or theUnited Kingdom or, generally, that offers items for consumption to atarget set of users that are able to understand English. Items forconsumption may include consumer products, services, electronic contentsuch as audio files, video files, electronic books, and electronicdocuments, access rights to content, and, generally, any items that maybe consumed. An entity operating the website 100 or another entity maywish to provide reviews to users (or other content) in languages otherthan English. For instance, an entity that operates the website of whichthe Web page 100 is a part may wish to make it easier for users that donot read English (or at least not well) to browse the Web page 100.Accordingly, the Web page 100 includes a selectable translation button108 to provide a translation of the review. An example of a Web page towhich the user may navigate in order to translate the review 102 isprovided in FIG. 5, which will be discussed in more detail below.

The translation button 108 may be provided to all users to whom thereview 102 is provided for display. Alternatively, the translationbutton 108 may be selectively or otherwise provided to users. Forinstance, the translation button 108 may be provided only to usersidentified in a data store as being able to communicate in a languageother than English or perhaps only to users identified in the data storeas having the ability to communicate in a particular language in which atranslation of the review 102 is desired. In this last example, forinstance, if the review 102 is not needed in Spanish, but is needed inRussian, the translation button 108 may be presented to users that areidentified in the data store as being able to communicate in Russian.Generally, any suitable way of selectively providing the translationbutton 108 may be used, and the manner in which the translation button108 is provided may vary depending on the context in which translationsare needed.

As noted, translations submitted by users using the translation button108 or otherwise provided may be used to provide corresponding contentin another language. Accordingly, FIG. 2 shows an illustrative exampleof a Web page 200 which includes content of the page 100 discussed abovein connection with FIG. 1 that has been translated. In this particularinstance, some of the content of the Web page 100 shown on FIG. 1 hasbeen translated into Russian for the Web page 200 shown in FIG. 2. Forexample, the Web page 200 includes a review 202 which, in this example,is a Russian translation of the review 102 shown in FIG. 1. The review202 in FIG. 2 may have been translated by an automated translation tooland/or by one or more users. In Web page 200, a yes button 204 and a nobutton 206 are provided to the user and which may operate similarly tothe yes button 104 and the no button 106 described above in connectionwith FIG. 1. Text appearing on the yes button 204 and no button 206 inFIG. 2, however, in this example, are translated to Russian for thebenefit of a user to whom the Web page is displayed.

In the Web page 200, an indication 208 is provided to the user in orderto inform the user that the review 202 was originally submitted inanother language. In this particular instance, the indication 208 is aphrase in Russian informing the user that the review 202 is atranslation of a review originally submitted in English. Also providedin the Web page 200, in this example embodiment, is a translationimprovement button 210 which is selectable by the user to correct anyportion of the translation or otherwise provide his or her owntranslation. While the translation improvement button 210 is shown withEnglish text, the text may be in Russian, as may other portions of theweb page 200 and/or an application used to display the web page 200.Selection improvement button 210 may allow the user to navigate to theWeb page shown in FIG. 5, or a page having similar features. The Webpage shown in FIG. 5, and features shown thereon, is described in moredetail below. In an embodiment, a user is allowed to change atranslation, although one or more processes for authorizing changes totranslations may be used. For instance, users may be provided an optionto vote on one or more changes of a translation and a determinationwhether the change may be made based at least in part on votes received.In this manner, incorrect translations may be avoided.

Also shown in the example embodiment of FIG. 2 is a portion of theoriginal content viewing element 212 that is selectable by a user inorder to view the translated content in its original language. In thisexample, the original content viewing element 212 is presented as textwith an embedded command that, upon selection of the text, causes thecontent to be displayed in the language it was originally submitted in.The command may be a hyperlink to a page containing the originalcontent, a JavaScript® command that causes the original content to bedisplayed, or, generally, any mechanism that can be used to cause theoriginal content to be provided to the user. The original contentviewing element 212, or other elements, may include other options, suchas the ability to view the content in a language other than the originallanguage, such as a language to which the original content was alsotranslated.

In some instances, users who may not understand another language wellenough to provide a translation may nevertheless recognize that atranslation is a poor translation. For instance, if a word is spelledwrong, grammar is incorrect and/or if spelling and grammar are correct,but nonstandard phrasing is used, the user may recognize thattranslation is poor. Similarly, if the review is simply difficult tounderstand, the user may recognize that the translation is poor. Forexample, a person who understands only Russian may recognize that atranslation is poor based solely on the translation itself, without anysubstantial knowledge of the language from which the translation wasmade. Of course, users who do understand other languages may alsorecognize the translation is poor. Accordingly, in an embodiment, theWeb page 200 includes a yes button 214 and a no button 216 that in thiscase allows the user to whom the Web page 200 is displayed to indicatewhether or not the user thinks that the translation is good. Forinstance, if the user thinks that the translation is good, he or she mayselect the yes button 214 and information regarding the user selectionof the yes button 214 may be responsibly communicated to an appropriateserver. Similarly, the user may select the no button 216 to indicatethat he or she thinks that the translation is poor and informationcorresponding to the user's selection would be transmittedappropriately. As will be described below, information regarding whetheror not users think the translation is good or bad is collected and usedin various ways.

As noted, the embodiment shown in FIG. 2 is provided for the purpose ofillustration and variations are contemplated as being within the scopeof the present disclosure. Accordingly, FIG. 3 shows an illustrativeexample of such a variation that may be used in accordance with anembodiment. In particular, FIG. 3 shows a Web page 300 in which contentfrom the Web page shown in FIG. 1 has been translated. The Web page 300may be provided as an alternative to the Web page 200 shown in FIG. 2.In this example, the Web page 300 is in many aspects similar to the Webpage 200 shown in FIG. 2. For example, the Web page 300 includes areview 302 which is a translation of the review 102 discussed above inconnection with FIG. 1. Returning to FIG. 3, the Web page 300 alsoincludes a yes button 304 and a no button 306 translated into Russian toallow the user to whom the Web page 300 is displayed to indicate whetheror not he or she finds that review 302 is helpful. The Web page 300 inthis instance also includes a pop-up box 308 which may appearsuperimposed on the Web page 300 subsequent to user input indicative ofuser intent to view the original text from which the review 302 wastranslated. The user input may be, for example, selection of the phrase“See original text.” For instance, the pop-up box 308 may appearsuperimposed on the Web page 300 subsequent to the user selection of thephrase “see original text.” The pop-up 308 in the embodiment includesthe original review 310 which, in this example, is the review 102discussed above in connection with FIG. 1.

In FIG. 3, the pop-up 308 in this example includes a yes button 312 anda no button 314, which may operate similarly to the yes button 214 andthe no button 216 discussed above in connection with FIG. 2. Thus, theuser may select either the yes button 312 or the no button 314 in orderto indicate whether or not the user thinks the translation 302 is a goodtranslation of the original translation 310 shown on the pop-up box 308.Because, in this example, the yes button 312 and no button 314 are inthe pop-up box 308, the user is only provided the ability to indicatewhether or not he or she thinks that translation of the original review310 to the review 302 is good when the user can view the original review310 in English with the translated review 302. Therefore, it is ensuredthat the user is able to compare the text of both the original review310 and the translated review 302 when selecting either the yes button214 or no button 216. However, variations are possible, includingproviding a yes button and a no button both directly on the web page 300and in the pop-up box 308. The Web page 300 and the pop-up box 308 also,in this example, include a translation improvement button 316 which mayoperate similar to the translation improvement button 210 describedabove in connection with FIG. 2.

FIG. 4 shows an illustrative example of another variation that may beprovided to users, in accordance with an embodiment. In FIG. 4, anotherWeb page 400 is shown. The web page 400 may be similar in severalaspects to the Web page 200 discussed above in connection with FIG. 2.For example, as shown in the figure, the Web page 400 includes a review402, a yes button 404 and a no button 406 that allow the user toindicate, as discussed above, whether or not the user thinks the reviewis helpful. In this example, however, only a portion of the review 402appears as a translation of the review 102 discussed above in connectionwith FIG. 1. For instance, the first sentence of the review 402 is shownin the language of the originally submitted review (English), while thesecond sentence of the review 402 is shown translated into Russian.Similarly, the third sentence of the review 402 is shown in English andthe fourth and fifth sentences of the review 402 are shown in Russian.There can be various reasons why the review 402 is shown in twodifferent languages.

For example, previously submitted reviews containing the phrase “thequality is terrible” in English may have already been translated intoRussian (by the system or other reviewers). If one or more of theseRussian translations of the phrase “the quality is terrible” isconsistent and/or if the Russian translation is highly rated (whereratings of reviews are discussed more below), then the prior Russiantranslation(s) of the phrase “the quality is terrible” may have bestored in a data store. Thus, when another review is submitted inEnglish with the phrase “the quality is terrible” a reliable Russiantranslation of the phrase is readily available. Translations of someportions of content may be provided based at least in part ontranslation consistency among users, whereas other portions ofcontinuity may be provided based at least in part on content having beentranslated by one or more highly rated translators. Generally, manydifferent ways of translating portions of content may be combinedtogether to form content. Thus, for instance, translations of differentsentences of the review 402 may have been provided in different ways andthe translations of the different sentences may be combined to form alarger piece of content. Further, instead of leaving content in theoriginal language, an automated translation tool may be used for contentfor which a reliable translation is not available.

As noted, in various embodiments, users are provided an opportunity tosubmit translations of content. Accordingly, FIG. 5 shows anillustrative example of a Web page 500 which may be provided to usersthat have indicated an intent to help with translation. The Web page 500may have been sent, for example, to any user that has indicated anintent to assist with translations or may be selectively provided tocertain users, such as users that have signed up to be translators, tousers that have demonstrated a level of proficiency for translating, andthe like. Also, variations the page 500 may be provided to users, suchas users identified as understanding more than one language in order toentice the users to sign up as translators of content. In thisillustrative example, the Web page 500 may be provided to a user thathas selected the translation button 108 in FIG. 1, the improvementbutton 210 shown in FIG. 2, the improvement button 316 shown in FIG. 3,or the translation improvement button 408 of FIG. 4. Generally, the Webpage 500 may be provided to a user subsequent to the receipt of anyindication of user intent and/or interest in assisting with translation,or in any suitable manner.

In this example, the Web page 500 includes content 502, which in thisexample, is identical to the review 102 shown in FIG. 1. The content 502is provided along with a translation input box 504, which an embodimentis an interface element that allows a user to type, paste or otherwiseinput a translation of the content 502. In this example, the originalcontent 502 is parsed into sentences, allowing a translator to inputtranslations for one or more of the sentences of the content 502. Asubmission button 506 allows a user to select the button in order tosend a completed translation to an appropriate server which receives thetranslation and forwards the translation appropriately, such as to acomputer system operable to store the translation in a data store. Ifthe user only provided translations for portions of the content 502,then the translated portions may be submitted upon selection of thesubmission button 506. A system that receives a submitted translationmay identify the language into which the content was translated usingvarious techniques. In one embodiment, the system detects the languageof the translation. Language abilities of users stored in user profilesmay be used to determine a language of a translation. Users may also beable to specify the language of the translation. For instance, the Webpage may include a drop-down box or other interface element that allowsthe user to select the language of the translation.

Also in this example, various other translations 508 of the content 502that were perhaps submitted by other users, or that have provided byvarious translation tools, or by a combination of other users andautomated translation tools may be provided to the user in the Web page500 in order to assist the user in preparing the translation or at leastto provide the user with examples of how content has been previouslytranslated. Thus, for instance, if the user is not certain how totranslate a particular phrase, he or she may see how others havetranslated the phrase and may use that phrase if he or she thinks thatthe translation of the phrase is correct. In this example, translationsby other translators are in Russian even though translations may havebeen received for other languages. In an embodiment, translations byother users (or by automated methods) may be provided to a user based atleast in part on the languages the user understands. Thus, in thisexample, the user is provided translations in Russian. However,translations by others may be provided in other languages. For example,translations received for all or some languages into which the content502 has been translated may be provided. In an embodiment, users areable to self select in a user interface, such as a profile creationand/or editing page, the languages he or she is able to or willing totranslate.

In addition, as shown in this example, the Web page 500 includes a yesbutton 510 and a no button 512 which may operate similarly to the yesbutton 214 and no button 216 in order to allow the user to indicatewhether he or she thinks that each of the other translations 508 isgood. In this manner, the votes of people participating in translationsmay be used to determine the quality of various translations. In otherwords, in this manner, the translators are provided an opportunity to bereviewers as well.

It should be noted that the Web pages shown in FIGS. 1-5 are providedfor the purpose of illustration and that variations are contemplated asbeing in the spirit of the present disclosure. For instance, FIGS. 1-5specifically show respective Web pages displayed utilizing a particularbrowser application, although other ways of displaying content to userswould be utilized. For example, an application dedicated to translationservices may be utilized in order to display content to users. Inaddition, the Web pages shown in FIGS. 1-5 show particular arrangementsof content interface elements and the like for the purpose ofillustration, but other arrangements may be used. For instance,generally, any interface elements that allow users to communicatetranslations and opinions thereof may be used. In addition, while theexamples shown in FIGS. 1-5 provide an illustrative example of contentbeing in the form of user-generated reviews, embodiments in the presentdisclosure may be used in order to provide translations of othercontent. For instance, any user-generated content may be translated inaccordance with various embodiments described herein. Generally, anytranslatable content may be translated using the embodiment describedand variations thereof. Examples of content that may be translatedinclude text in one or more languages, audio files (or portions thereof)having spoken words in one or more languages, video files (or portionsthereof) with audio tracks having spoken words in one or more languagesand/or subtitles in one or more languages, video files showing one ormore visual languages, such as American Sign Language, and, generally,any content that is used for expression in one or more languages.

FIG. 6 shows an illustrative example of an environment 600 in whichvarious embodiments of the present disclosure may be implemented. Theenvironment 600 may be used, for instance, to communicate the Web pagesshown in FIGS. 1-5, variations thereof, and user interactions therewith.In this example, a user device 602 communicates over the Internet 604with a Web server 606 in order to access various Web pages of thewebsite provided by Web server 606 and to communicate interactiontherewith. The user device 602 may be any device that may be used inorder to view, hear, or otherwise access content. The examples include,but are not limited to, desktop computers, notebook computers, mobiledevices, electronic book readers, tablet-computing devices and generallyany computing device with which content can be provided to users.Further, while the environment 600 is shown as including the Internet604, other communications networks may be used, including any suitablenetwork for communicating information, such as an internal network of anorganization or a mobile network and/or combinations of networks.

In the environment 600 as shown in FIG. 6, the Web server 606 accesses acontent data store system 608 in order to provide content to the user602. The content data store 608 may include, for example, hypertextmarkup language (HTML) files, extensible markup language (XML) filesand/or generally data records that contain content. The content can beany content which is used to express information in a language, such asany of the various types of content discussed above.

The Web server 606 also, in an embodiment, accesses a translation datastore 610 which, in an embodiment, includes translations of some of thecontent in the content data store 608 and/or portions thereof. Forinstance, using the example of FIGS. 1-5, user-generated reviews thathave been translated using various means may be stored in thetranslation data store 610. Data in the translation data store 610 maybe organized in a manner associating translations with content of whichthe translations are translated. With the example of FIGS. 1 and 2, forinstance, the review 202 of FIG. 2 may be stored in the translation datastore 610 in a manner that associates the translation 202 of FIG. 2 withthe review 102 of FIG. 1. One or more relational database tables may beused to associate translations with corresponding content. Generally,data in the translation data store 610 may be stored in any manner thatallows for location of translations of corresponding content. Data inthe translation data store may also be stored in a manner thatidentifies the language of the translation. Thus, for instance, if thereview 102 of FIG. 1 has been translated into multiple languages, anappropriate translation may be readily available from the translationdata store 610 of FIG. 6.

The specific arrangement in FIG. 6 is provided for the purpose of thisillustration and other environments may be utilized in accordance withvarious embodiments. For instance, the figure of FIG. 6 shows a singleuser device 602, although, generally, multiple-user devices, which maybe of different types, may utilize the Internet 604 or othercommunications network or combination of network to access the Webserver 606. As another example, multiple Web servers may access thecontent of content data store 608 and the translation data store 610. Inaddition, for the purpose of illustration, only certain components ofthe environment 600 are shown and other components are not shown. Forinstance, a website may be provided collectively by multiple servers andnot just a single Web server 606. Similarly, each of the content datastore 608 and the translation data store 610 may comprise multiple datastores or be implemented in a single data store. The content data store608 and translation data store 610 may also be accessed by the Webserver 606 over the Internet or other network. Generally, othervariations are also contemplated as being in the spirit of thedisclosure.

FIG. 7 shows a flowchart for an illustrative example of a process 700for providing content in accordance with an embodiment. The illustrativeprocess 700 may be used to provide translations of content to users whenappropriate and when such translations are available, as discussed morebelow. Portions of the process 300 (or any other processes describedherein or variations and/or combinations thereof) may be performed underthe control of one or more computer systems configured with executableinstructions and may be implemented as code (e.g., executableinstructions of one or more computer programs, or one or moreapplications executing collectively on one or more processors, byhardware, or combinations thereof). The code may be stored on acomputer-readable storage medium; for example, in the form of a computerprogram comprising a plurality of instructions executable by one or moreprocessors.

As noted, the process 700, or variations thereof, may be used to, inappropriate situations, identify and provide appropriate translations ofcontent to present to a user. In an embodiment, the request for contentis received 702 in accordance with an embodiment. The request forcontent may be received in any suitable manner. For instance, therequest for content may be a hypertext transfer protocol (HTTP) requestsent by a user device in connection with a Web browser executing on theuser device. The request may be sent over a communications network to aserver operable to receive the HTTP request and respond with contentaccordingly. Generally, however, the request for content may be made inother ways. In an embodiment, once the content is received, adetermination is made 704 whether an identity of a user associated withthe request for content is ascertained. The determination whether theuser's identity is ascertained may be performed in any suitable manner.For instance, a cookie associated with the user may have been sent froma computing device of the user and used in order to identify the userfrom a data store that associates cookies with user accounts. The useralso may have provided identification information, such as during anauthentication process utilizing a username and password input by theuser. Generally, any method of identifying the user and/or determiningwhether or not the user is identified may be used.

In an embodiment, if the identity of the user is not ascertained, thencontent is provided 706 in a default language. For instance, in anelectronic commerce website dedicated to offering items for consumptionin North America, website content may be provided in English. Similarly,the default language for a website primarily directed to users in Russiaor to Russian speakers in another location may be in Russian. Generally,the default language may be in any language. If, however, the user'sidentity is ascertained, then a determination is made 708 whether aprimary language of the user is known. For instance, a user may have auser profile stored in connection with the website from which contentwas requested. User profile may identify the primary language of theuser. Similarly, information about the user may be used to deduce aprimary language of the user. For instance, if it is known that the uservisits many websites that are in a particular language, that particularlanguage may be assumed to be a primary language of the user. Similarly,if the request for content was received from a Web page of a particularlanguage, that particular language may be assumed to be the primarylanguage of the user. Information about the website from which the usernavigated may be ascertained, for instance, from an HTTP referrer valueprovided with the request for content discussed above, in an embodiment.In any event, if the primary language of the user is not known, in anembodiment, then content in the default language is provided 706 inaccordance with the embodiment.

If the primary language is known, however, then a determination is made710 whether the primary language is the same as the default language. Ifthe primary language is the same as the default language, then, in anembodiment, a determination is made 712 whether the user understands asecond language. If the primary language is the default language and asecond language is not known for the user, then, in an embodiment,content is provided 706 in the default language. Determination ofwhether the user communicates or otherwise understands the secondlanguage may be performed in a suitable manner, such as by referencing aprofile of the user or by deducing or otherwise assuming the userunderstands a second language based on other information known about theuser, such as a language on the website from which the user hasnavigated.

If the primary language of the user is a default language and it isdetermined that the user understands a second language, then in anembodiment, content is provided 714 in the default language with atranslation option. Providing content with a translation option may bedone in any suitable manner. In an embodiment, providing content with atranslation option includes providing to the user content with aninterface element that is selectable by the user for allowing the userto navigate to a page in which the user may provide a translation. Forinstance, the translation button 108 of FIG. 1, or a similarlyfunctional interface element, may be provided with content provided tothe user. In an embodiment, if the primary language is not a defaultlanguage, however, a determination is made 716, nevertheless, whetherthe user understands a second language. If a determination is made 716that the user understands the second language in an embodiment, adetermination is made 718 whether there is content available for theuser's primary language, such as whether a translation of the requestedcontent or a portion thereof is stored in the data store. If content isnot available for the user's primary language, then in an embodiment,content is provided 714 in the default language with a translationoption, such as in the manner described above. If, however, there iscontent available for the user's primary language and it is determinedthat the user understands a second language, then the content isprovided 720 in the primary language with the translation option. If therequested content was for a particular Web page, for instance, atranslation into the user's primary language may be provided to the userwith a translation option in accordance with the embodiment. If,however, it is determined that the user does not understand the secondlanguage or it is simply not known that the user understands a secondlanguage, a determination is made 722 whether there is content availablefor the primary language. If content is not available for the primarylanguage, then content is provided in the default language 706 which maybe with or without translation function. If, however, content isavailable for the user's primary language, then in an embodiment,content is provided in the primary language 724 without a translationoption. Thus, if the requested content is a Web page, a translation ofthe Web page may be provided to the user without a translation option inaccordance with an embodiment.

Of course, the process illustrated in FIG. 7 includes specific actionsfor the purpose of illustration and variations of the methods 700 arepossible. For instance, a translation option may be provided to the userregardless of any determinations about the user's ability to understandanother language, whether or not such determinations are made. Also,more sophisticated criteria than that described above may be utilized inorder to determine the language in which content should be provided tothe user and/or whether or not to provide a translation to the user withthe provided content. As another example, the process 700 describedabove may be modified to take into account users who speak multiplelanguages. For users that speak more than two languages, for example,the process 700 may include selecting content in a language selectedfrom multiple languages understood by the users. Rankings of languagesmay be stored for users that understand multiple languages and, for auser, content in the highest ranked language that is available may beprovided to the user. Rankings may be based at least in part on userinput indicating a ranking and/or may be derived from user activity inconnection with content in various languages. As an example, if a useroften navigates to pages with content in Ukrainian, but less oftennavigates to pages with content in Russian, Ukrainian may be rankedhigher for the user than Russian and, if available, content in Ukrainianmay be selected over content in Russian. Other variations are consideredwithin the scope of the present disclosure.

In some instances, it may be desirable to limit the abilities of usersto provide translations. It may be desirable, for example, toselectively provide options for translating content to users so thatusers' efforts are more likely to be spent translating content that isin greater need of translation. Accordingly, FIG. 8 shows a flowchartfor an illustrative example of a process 800 that may be used in orderto selectively provide translation options as content in order toincrease the chance of any content being translated for content in whichthere is a greater need for translation. The process of FIG. 8 and thefollowing description describes the process 800 in terms of userreviews, such as those shown in FIGS. 1-5, although the process 800 maybe applicable to any translatable content. In this particular example ofFIG. 8, the process 800 is provided in connection with the examplesshown in FIGS. 1-5 where an entity operates a website to offer items forconsumption in another language, such as by launching a new website inanother country or simply by providing translated content on anoriginally existing website. Variations of the process 800 may be usedin various other contexts.

In an embodiment, an item on a first website in a first language isidentified 802. For instance, the process 800 may be performedrepeatedly for all items offered for consumption on a website. Each itemoffered for consumption on the website may be identified sequentially.Identification of an item for consumption may also be performed in otherways, such as in ways that are based upon user input indicating user'sintent on identifying the item. For instance, an administrator or even auser may identify through user input the item offered for consumption.As another example, identification of an item may occur when a user insome manner indicates that a translation of content related to the itemwould be helpful, such as by making an appropriate selection through awebsite interface.

In an embodiment, once the item is identified, a determination is made804 whether more reviews for the item are needed in the first language.Thus, for instance, if the first language is Russian, a determinationmay be made whether more reviews are needed in Russian in order toprovide users with a robust experience. A robust experience may beprovided to users, for instance, when the number of reviews availablefor an item in a language of the users exceeds a threshold. In anembodiment, if more reviews are needed in the first language, then adetermination is made 806 whether there are reviews for the identifieditem on a second website in a second language. Thus, continuing the sameexample, if the identified item is offered for consumption on a Russianwebsite, a determination may be made whether reviews for that item areavailable on a second, English-language website. Of course, otherlanguages may be used. If reviews for the identified item forconsumption are available, then in an embodiment, the reviews of thesecond website in the second language are provided 808 on both websiteswith a translation option. In this manner, reviews are available toviewers on a first website, albeit in a second language. Also in thismanner, if visitors to the first website understand the second language,they have the benefit of reading the reviews and, in an embodiment, theoption of translating reviews for other users. Similarly, visitors tothe second website are provided the option of translating the reviews,thereby making translations of the reviews available on the firstwebsite.

With the reviews on both websites provided with translation options, thetranslations are received 810 in accordance with an embodiment.Translations may be received, for instance, in connection with userinterface, such as Web page, similar to the Web page 500 described abovein connection with FIG. 5 or, generally, any suitable interface or inany suitable manner. Once translations are received in an embodiment,reviews in the second language on the first website that have beentranslated are converted 812 to their translations in accordance withthe embodiment. Thus, visitors of the first website may see translationsof the original reviews, such as in the manner described above inconnection with FIGS. 1-4.

Variations of the process 800 described above are contemplated as beingwithin the scope of the present disclosure. For instance, for thepurpose of illustration, FIG. 8 shows simply providing the reviews ofthe second website on both websites with the translation option.However, more sophisticated processes may be used. For instance,translation options may be only provided to users who satisfy certaincriteria, such as users identified as understanding the second languageor identified as being able to translate into the first language. Inaddition, one or more criteria may be used in order to determine when toreplace reviews in the second language on the first website withtranslations. For example, translations of the reviews may be providedon the website in the first language only responsive to conditionsindicative of the reviews being reliable being satisfied and/orresponsive to user request for translations. For instance, a uservisiting the first website may see a review in the second language inproximity to an option for requesting a translation of the review eventhough a translation for the review has been received. In other words,the translated review is not provided to the user automatically becausethe review has not been identified as being reliable. However, if theuser requests the translation, the review not identified as reliable maybe provided to the user. Of course, other variations are contemplated asbeing within the spirit of the present disclosure.

As discussed, translations, reviewers, and/or translators may be ratedin accordance with various embodiments. FIG. 9 shows a flowchart for anillustrative example of a process 900 that may be used in order to ratetranslations and reviewers of translations. A translator in anembodiment is a user, computing device, or application operating on acomputing device that translates content. For instance, in anembodiment, the translator is a person that manually providestranslations by composing translations and submitting translationsthrough an interface. In an embodiment, the process 900 includesproviding 902 translated content. Translated content may be provided inany manner, such as in the manner described above in connection withFIGS. 2-4 or in other ways, such as through the application and even inphysical ways, such as by providing the physical printout of translatedcontent. Once the translated content is received, in an embodiment, atranslation quality vote is received 904 from a reviewer. A translationquality vote in an embodiment is an indication from a user about theuser's opinion of the quality of the translation. For instance,referring to FIG. 2, a quality vote may be received in accordance withuser selection of the yes button 214 or the no button 216 shown in thefigure. In an embodiment, a quality vote is a value that indicates auser's opinion whether the translation is good or bad, although othervalues, such as values in a numerical range may be used. In anembodiment, when the quality vote is received, information regardingpast votes for the translation is access 906. A record in a data storecorresponding to the translation may be accessed. The record may includeinformation of the past votes, including statistics, such as how manyvotes the translation received, the types of votes received (yes, no,good, bad, and the like), ratings of users that submitted the votes, theidentities of users that submitted the votes, and/or generally anyinformation relevant to the voting process described herein.

In an embodiment, a determination is made 908 whether the receivedtranslation quality vote matches past votes received for thetranslation. For instance, if a translation has received primarily goodvotes and the received translation quality vote is a bad vote, thendetermination may be made if there is not a match. Similarly, if pastvotes have primarily been good votes and the translation quality vote isalso a good vote, then a determination may be made that receivedtranslation quality votes matches. More sophisticated ways ofdetermining whether there is a match may also be used. For instance, ina system where users rank translations on more refined scales (forexample votes that rank translations on a scale, such as a number ofstars out of five stars), a distance from an average ranking may be usedto determine whether there is a match. For example, if a scale from oneto five is used, with five indicating the highest ranking a voter cangive a translation, the difference between a vote for a translation andthe average of votes already received for the translation may be used todetermine whether the vote matches the past votes. If the difference isgreater than two, then it may be considered that there is not a match.Thus, if the average ranking for a translation is four and a half and avote for a ranking of two is received, it may be determined that thereis a match. However, if the vote indicated a ranking of three, then itmay be determined that there is a match. Other scales and methods ofdetermining whether there is a match may be used.

In an embodiment, if there is a match between the received translationquality vote and the past votes, then a determination is made 910whether the received translation quality vote is a good vote or a badvote. If the received translation quality vote is a good vote, then atranslator reputation score for a translator that provided thetranslation quality vote is adjusted 912 upward in accordance with anembodiment. A translator reputation score may be any score that isindicative of a corresponding translator's quality. For instance, for aparticular translator, a translator reputation score may be:

$T = \frac{\sum\limits_{t}{t\;{s(t)}}}{\sum\limits_{t}1}$where T represents the translator's translator reputation score, trepresents a translation provided by the translator, and ts(t)represents a translation score of the translation score t. Thus, in thisexample, the translation score T is an average of the translation scoresof translations provided by the translators. Other translation scoresmay be used. For instance, the sums shown in the example translationscore may be weighted by reviewer quality such that translationsreviewed by higher quality reviewers will affect the score more thantranslations reviewed by lower quality reviewers.

In addition, in an embodiment, if there is a match between the receivedtranslation quality vote and the past votes and the translation qualityvote is a good vote, a translation score for the translation itself isadjusted 914 upward and a reviewer reputation score for the reviewerthat submitted the received translation quality vote is adjusted 916upward as well. A translation score for a translation may be any scorethat indicates a quality of the translation. For instance, a translationscore of a translation t may be:

${t\;{s(t)}} = \frac{\sum\limits_{v}{{q(v)}{r(v)}}}{\sum\limits_{r}1}$where ts(t) represents the translation score (which may be the same asor different from the translation score described above) for atranslation t, v represents a vote submitted for the translation t, q(v)is a function that has a value of one if v is a good vote and a value ofzero if v is a bad vote, and r(v) is a reviewer score for the reviewerthat submitted the vote v. In this manner, the translation scorerepresents the sum of the good votes for the translation, weighted bythe reviewer score, divided by the total number of votes received forthe translation. Other translation scores may be used as well. Forexample, in the above equation for the translation score ts(t), q(v) maytake on a value of negative one or another value for bad votes. Asanother example, the translation score may simply be the number of goodvotes received for the translation divided by the total number of votesreceived for the translation (whether good or bad).

The reviewer score may be any score that indicates a quality of areviewer. For example, a reviewer score may be determined as:

$R = \frac{\sum\limits_{r}{a(r)}}{\sum\limits_{r}{v(r)}}$where R represents the reviewer score for a reviewer, r represents areview submitted by the reviewer, a(r) represents the number of usersthat have reviewed the translation of the review r and that agree withthe review r, and v(r) represents the total number of votes received fora translation reviewed by the review r. In this example, therefore, areviewer's reviewer score is the number of reviews by other users thatagree with the reviewer's reviews divided by the total number of votesfor translations that the reviewer has reviewed. Other reviewer scoresmay be used. For instance, the sums shown may be weighted by one or morefactors, such as translator scores of reviewers with which a particularreview does or does not agree.

In this manner of adjusting the translator score and translation score,scores representative of a translator's reputation, the translation'squality, and a reviewer's reputation are all increased because thereviewer and past voters agree that the translation is good. Also,because the reviewer agrees with other reviewers, the reviewer has hisor her reputation score increased because of the agreement with otherreviewers.

In an embodiment, if there is a match between the received translationquality vote and the past votes, but the received translation qualityvote was a bad vote, then the translator reputation score is adjusted918 downward, the translation score is adjusted 920 downward, and thereputation reviewer score is adjusted 922 upward. In this manner, ascore indicative of the translator's reputation is adjusted downwardbecause both the reviewer and past reviewers agree that the translationis bad. Similarly, because the reviewer and past reviewers agree thatthe translation is bad, the translation score is adjusted downward. Butbecause the reviewer agree with past reviewers, the reviewer'sreputation score is adjusted upward.

If in an embodiment there is not a match between the receivedtranslation quality vote and past votes, a determination is made 924whether the received translation quality vote was good or bad. If thereceived translation quality vote was good, then the translatorreputation score is adjusted 926 upward, the translation score isadjusted 928 upward and the reviewer reputation score is adjusted 930downward. The translator reputation score is adjusted downward because,collectively, the votes indicate less agreement on the quality of thetranslator of this particular translation done by the translator.Similarly, the translation score is adjusted upward because the receivedtranslation quality vote is good, which is in disagreement with the pastvotes, indicates that the translation is not as bad as the past voteshad indicated. In other words, the translation score is adjusted upwardbecause there is less agreement among users that the translation is bad.The reviewer score in this instance is adjusted downward because thereviewer does not agree with other reviewers.

In an embodiment, if there is not a match between the receivedtranslation quality vote and the past vote, and the received translationquality vote is a bad vote, then in an embodiment, the translatorreputation score is adjusted 932 downward, the translation score isadjusted 934 downward and the reviewer reputation score is adjusted 936downward. In this manner, the translator reputation score is adjusteddownward because there is less agreement about the translation of thetranslator being good. Similarly, the translation score is adjusteddownward because there is less agreement among users that thetranslation itself is good and the reviewer's reputation score isadjusted downward because the reviewer did not agree with pastreviewers.

The process 900 may be performed in connection with a translation everytime a vote regarding the translation's quality is received inaccordance with an embodiment. Variations of the process 900 are alsocontemplated as being in the spirit of the present disclosure. Forinstance, multiple votes may be received before translator reputationscores, translation scores and reviewer reputation scores are adjusted.In this manner, more information is collected from users beforeconclusions are made about the translation's quality. In addition, whileprocess 900 shows the reviewer's reputation score being adjusted, allreviewers having reviewed the translations may have their scoresadjusted as well. For instance, past reviewers of the translation mayhave their scores adjusted upward when the current reviewer agrees aboutthe translation's quality. Similarly, past reviewers may have theirscores adjusted downward when the current reviewer disagrees with thetranslation's quality. In this manner, if reviewers initially agreeabout a translation, but then more reviewers come along and disagree,then the reviewers that disagree will ultimately cause scores to beadjusted and corrected accordingly.

As noted, when one reviewer of the translation votes, the scores ofmultiple reviewers may be adjusted due to the fact that the statisticsregarding agreement among reviewers has changed due to the receivedvote. Accordingly, FIG. 10 shows a flowchart for an illustrative exampleof a process 1000 that may be used to adjust reviewer scores inaccordance with the embodiment. In an embodiment, the process 1000includes receiving 1002 a translation quality vote from a reviewer. Thetranslation quality vote may be received such as in a manner describedabove. In addition, information regarding past translation quality votesis access 1004 in accordance with the embodiment, such as in the mannerdescribed above. When the information regarding the past translationvotes is accessed in an embodiment, past reviewers are identified 1006.Past reviewers may be identified so that the scores for the pastreviewers may be adjusted according to the nature of the receivedtranslation quality vote.

In an embodiment, a determination is made 1008, based at least in parton the received translation quality vote and the accessed informationregarding past votes, whether there is a match between the receivedtranslation quality vote and the past votes. If there is a match betweenthe received translation quality vote and the past votes, then in anembodiment, all reviewer's reputation scores are adjusted 1010 upwardaccordingly. However, if in an embodiment, there is not a match betweenthe received translation quality vote and the past votes, then adetermination is made 1012 whether the current reviewer is morereputable than the past reviewers. Determining whether the currentreviewer is more reputable than the past reviewers may be accomplishedin any suitable manner. For instance, in an embodiment, determiningwhether the current reviewer is more reputable than the past reviewer isperformed by comparing a reviewer reputation score of the currentreviewer with an average of the reviewer reputation score of the pastreviewers. More sophisticated ways of determining whether the currentviewer is more reputable than the past reviewer may be used. Forinstance, in an embodiment, if the past reviewers include a thresholdnumber of reviewers that have corresponding reputation scores exceedinga reputation score threshold, those reviewers having the reputationscores exceeding the reputation score threshold may be compared with thecurrent reviewer. If user reputation scores do not exceed reputationscore threshold, they are not considered or are considered with lessweight. In this manner, a group of highly rated translators are used asa basis for comparison in order to prevent situations such as whenthere's general agreement among past reviewers about the translation,but the average reputation score is low for various reasons. Forinstance, there may be many reviewers with low reputation scores, butnevertheless agree with just a few reviewers having high reputationscores. In other words, low reputation scores by themselves may not beindicative of a reviewer's quality, but may simply be due to havingperformed few activities as a reviewer.

In any event, in an embodiment, if it is determined that the currentreviewer is more reputable than the past reviewer, then the pastreviewer's scores are adjusted 1014 downward in accordance with anembodiment. If, however, the past reviewer is determined to be morereputable than the current reviewer, then the current reviewer's scoreis adjusted 1016 downward. In this manner, only the less reputable ofthe reviewer or past reviewers have their corresponding reviewerreputation score(s) adjusted downward, thereby preventing a number ofbad reviewers affecting the score of a good reviewer.

Variations of the process 1000 are considered as being within the scopeof the present disclosure. In addition, variations and combinations ofthe processes described above are also considered as being within thescope of the present disclosure, including variations described above inconnection with other processes. For instance, scores are adjusted asvotes are received; however, users, whether translators or reviewers,may have their own reputation scores changed over time. Thus, in anembodiment, data that is collected recording who has voted for whichtranslation and reputation scores and translation scores may be adjustedaccordingly as user reputation scores change over time. As an example,if a reviewer becomes a highly rated reviewer, then that reviewer's pastactions may be provided more weight in recalculating scores fortranslation and reviewers connected with the reviewer having the highreputation score. Similarly, if a user initially has a high reputationscore, but his or her score diminishes over time, the effect that userhas had in the past may also diminish over time because of therealization that the reviewer should have less influence than what wasinitially given. As yet another example, one or more quality-controlmechanisms may be in place to enhance the quality of scores computed inaccordance with various embodiments. Votes, for instance, may be checkedfor uniqueness to prevent users from voting multiple times on the sametranslation. Users may be prevented from voting multiple times on thesame translation and/or multiple votes from a single user for the sametranslation may be disregarded or at least provided less weight incalculations of scores. Similarly, users may be prevented from votingfor translations that they themselves submitted. As above, users maysimply be prevented from voting on their own translations and/or votesfrom a user for his or her own translation may be disregarded or givenless weight.

Embodiments of the present disclosure may be used to providetranslations to various entities. FIG. 11 shows an illustrative exampleof an environment 1100 which may be used in order to provide content inaccordance with an embodiment. The components of the environment 1100may implement one or more of the embodiments described herein orvariations thereof. In this particular example, a user device 1102accesses a website 1104, such as in the manner described above. Anoperator of the website 1104 may wish to provide translated content to auser of the user device 1102 and therefore may utilize a translation webservice 1106. A translation web service in an embodiment is a webservice that receives requests for translations and provides translationin response, either immediately or when translations become available.The website 1104 and the translation 1106 may be operated by the sameentity or by different entities. For instance, the translation webservice 1106 may provide services to many websites 1104 that are in needof translated content. In an embodiment, the translation web service1106 provides an application programming interface (API) that iscallable by devices utilized in connection with the website 1104. Forinstance, the website 1104 may send a call to the translation webservice 1106. The call may include the content to be translated, oridentification thereof, and an identification of a language or multiplelanguages into which the content should be translated. The call may alsoinclude the identification of the language of the content that wasprovided, although the language of the content that was provided may bedetected by the translation web service.

When the translation web service 1106 receives a request for atranslation in an embodiment, the translation web service acts as atranslation data store 1108, which includes past translation. Thecontent to be translated may be parsed and if there are strings or otherportions of the content already translated in the translation data store1108, then a translation of those portions may be accessed. All of thecontent or the remaining portion of the content may be translated in anysuitable manner, such as in a manner described above. For instance, thecontent may be provided on a website, such as the website 500 shown inFIG. 5 for a user to translate. In addition to the foregoing, thetranslation web service 1106 may be operated in connection with its ownwebsite. For instance, as shown in the diagram, the translation webservice communicates with various websites 1110 in different languages.Websites may be translations of an electronic commerce website operatedin connection with the translation web service. Thus, the translationweb service 1106 may provide translations to its own websites, as wellas its third-party websites, such as the website 1104. The translationweb service may or may not request a fee for translations provided tothird parties. The translation web service may simply translate contentfor others in order to improve the data in the translation data store1108.

FIG. 12 illustrates an example of an environment 1200 for implementingaspects in accordance with various embodiments. FIG. 12, for example,illustrates various components that may be used in the environmentsdescribed above and/or that may be used to implement the varioustechniques described above. As will be appreciated, although a Web-basedenvironment is used for purposes of explanation, different environmentsmay be used, as appropriate, to implement various embodiments. Theenvironment includes an electronic client device 1202, which can includeany appropriate device operable to send and receive requests, messages,or information over an appropriate network 1204 and convey informationback to a user of the device. Examples of such client devices includepersonal computers, cell phones, handheld messaging devices, laptopcomputers, set-top boxes, personal data assistants, electronic bookreaders, and the like. The network can include any appropriate network,including an intranet, the Internet, a cellular network, a local areanetwork, or any other such network or combination thereof. Componentsused for such a system can depend at least in part upon the type ofnetwork and/or environment selected. Protocols and components forcommunicating via such a network are well known and will not bediscussed herein in detail. Communication over the network can beenabled by wired or wireless connections, and combinations thereof. Inthis example, the network includes the Internet, as the environmentincludes a Web server 1206 for receiving requests and serving content inresponse thereto, although for other networks an alternative deviceserving a similar purpose could be used as would be apparent to one ofordinary skill in the art.

The illustrative environment includes at least one application server1208 and a data store 1210. It should be understood that there can beseveral application servers, layers, or other elements, processes, orcomponents, which may be chained or otherwise configured, which caninteract to perform tasks such as obtaining data from an appropriatedata store. As used herein the term “data store” refers to any device orcombination of devices capable of storing, accessing, and retrievingdata, which may include any combination and number of data servers,databases, data storage devices, and data storage media, in anystandard, distributed, or clustered environment. The application servercan include any appropriate hardware and software for integrating withthe data store as needed to execute aspects of one or more applicationsfor the client device, handling a majority of the data access andbusiness logic for an application. The application server providesaccess control services in cooperation with the data store, and is ableto generate content such as text, graphics, audio, and/or video to betransferred to the user, which may be served to the user by the Webserver in the form of HTML, XML, or another appropriate structuredlanguage in this example. The handling of all requests and responses, aswell as the delivery of content between the client device 1202 and theapplication server 1208, can be handled by the Web server. It should beunderstood that the Web and application servers are not required and aremerely example components, as structured code discussed herein can beexecuted on any appropriate device or host machine as discussedelsewhere herein.

The data store 1210 can include several separate data tables, databases,or other data storage mechanisms and media for storing data relating toa particular aspect. For example, the data store illustrated includesmechanisms for storing production data 1212 and user information 1216,which can be used to serve content for the production side. The datastore also is shown to include a mechanism for storing log data 1214,which can be used for reporting, analysis, or other such purposes. Itshould be understood that there can be many other aspects that may needto be stored in the data store, such as for page image information andto access right information, which can be stored in any of the abovelisted mechanisms as appropriate or in additional mechanisms in the datastore 1210. The data store 1210 is operable, through logic associatedtherewith, to receive instructions from the application server 1208 andobtain, update, or otherwise process data in response thereto. In oneexample, a user might submit a search request for a certain type ofitem. In this case, the data store might access the user information toverify the identity of the user, and can access the catalog detailinformation to obtain information about items of that type. Theinformation then can be returned to the user, such as in a resultslisting on a Web page that the user is able to view via a browser on theuser device 1202. Information for a particular item of interest can beviewed in a dedicated page or window of the browser.

Each server typically will include an operating system that providesexecutable program instructions for the general administration andoperation of that server, and typically will include a computer-readablemedium storing instructions that, when executed by a processor of theserver, allow the server to perform its intended functions. Suitableimplementations for the operating system and general functionality ofthe servers are known or commercially available, and are readilyimplemented by persons having ordinary skill in the art, particularly inlight of the disclosure herein.

The environment in one embodiment is a distributed computing environmentutilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 12. Thus, the depiction of the system 1200 in FIG.12 should be taken as being illustrative in nature, and not limiting tothe scope of the disclosure.

The various embodiments further can be implemented in a wide variety ofoperating environments, which in some cases can include one or more usercomputers, computing devices, or processing devices which can be used tooperate any of a number of applications. User or client devices caninclude any of a number of general purpose personal computers, such asdesktop or laptop computers running a standard operating system, as wellas cellular, wireless, and handheld devices running mobile software andcapable of supporting a number of networking and messaging protocols.Such a system also can include a number of workstations running any of avariety of commercially-available operating systems and other knownapplications for purposes such as development and database management.These devices also can include other electronic devices, such as dummyterminals, thin-clients, gaming systems, and other devices capable ofcommunicating via a network.

Most embodiments utilize at least one network that would be familiar tothose skilled in the art for supporting communications using any of avariety of commercially-available protocols, such as TCP/IP, OSI, FTP,UPnP, NFS, CIFS, and AppleTalk. The network can be, for example, a localarea network, a wide-area network, a virtual private network, theInternet, an intranet, an extranet, a public switched telephone network,an infrared network, a wireless network, and any combination thereof.

In embodiments utilizing a Web server, the Web server can run any of avariety of server or mid-tier applications, including HTTP servers, FTPservers, CGI servers, data servers, Java servers, and businessapplication servers. The server(s) also may be capable of executingprograms or scripts in response requests from user devices, such as byexecuting one or more Web applications that may be implemented as one ormore scripts or programs written in any programming language, such asJava®, C, C# or C++, or any scripting language, such as Perl, Python, orTCL, as well as combinations thereof. The server(s) may also includedatabase servers, including without limitation those commerciallyavailable from Oracle®, Microsoft®, Sybase®, and IBM®.

The environment can include a variety of data stores and other memoryand storage media as discussed above. These can reside in a variety oflocations, such as on a storage medium local to (and/or resident in) oneor more of the computers or remote from any or all of the computersacross the network. In a particular set of embodiments, the informationmay reside in a storage-area network (“SAN”) familiar to those skilledin the art. Similarly, any necessary files for performing the functionsattributed to the computers, servers, or other network devices may bestored locally and/or remotely, as appropriate. Where a system includescomputerized devices, each such device can include hardware elementsthat may be electrically coupled via a bus, the elements including, forexample, at least one central processing unit (CPU), at least one inputdevice (e.g., a mouse, keyboard, controller, touch screen, or keypad),and at least one output device (e.g., a display device, printer, orspeaker). Such a system may also include one or more storage devices,such as disk drives, optical storage devices, and solid-state storagedevices such as random access memory (“RAM”) or read-only memory(“ROM”), as well as removable media devices, memory cards, flash cards,etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.), and working memory asdescribed above. The computer-readable storage media reader can beconnected with, or configured to receive, a computer-readable storagemedium, representing remote, local, fixed, and/or removable storagedevices as well as storage media for temporarily and/or more permanentlycontaining, storing, transmitting, and retrieving computer-readableinformation. The system and various devices also typically will includea number of software applications, modules, services, or other elementslocated within at least one working memory device, including anoperating system and application programs, such as a client applicationor Web browser. It should be appreciated that alternate embodiments mayhave numerous variations from that described above. For example,customized hardware might also be used and/or particular elements mightbe implemented in hardware, software (including portable software, suchas applets), or both. Further, connection to other computing devicessuch as network input/output devices may be employed.

Storage media and computer readable media for containing code, orportions of code, can include any appropriate media known or used in theart, including storage media and communication media, such as but notlimited to volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information such as computer readable instructions, data structures,program modules, or other data, including RAM, ROM, EEPROM, flash memoryor other memory technology, CD-ROM, digital versatile disk (DVD) orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other medium which canbe used to store the desired information and which can be accessed bythe a system device. Based on the disclosure and teachings providedherein, a person of ordinary skill in the art will appreciate other waysand/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit theinvention to the specific form or forms disclosed, but on the contrary,the intention is to cover all modifications, alternative constructions,and equivalents falling within the spirit and scope of the invention, asdefined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including,”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected” is to be construed as partly or wholly contained within,attached to, or joined together, even if there is something intervening.Recitation of ranges of values herein are merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range, unless otherwise indicated herein, and eachseparate value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (e.g., “such as”) provided herein, isintended merely to better illuminate embodiments of the invention anddoes not pose a limitation on the scope of the invention unlessotherwise claimed. No language in the specification should be construedas indicating any non-claimed element as essential to the practice ofthe invention.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the invention.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Theinventors expect skilled artisans to employ such variations asappropriate, and the inventors intend for the invention to be practicedotherwise than as specifically described herein. Accordingly, thisinvention includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the invention unlessotherwise indicated herein or otherwise clearly contradicted by context.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the disclosure asset forth in the claims.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

For the Examiner's convenience, Applicants note that this application isa continuation of U.S. application Ser. No. 12/881,605. The claims ofthe present application are different and possibly, at least in someaspects, broader in scope than the claims pursued in the parentapplication. To the extent any prior amendments or characterizations ofthe scope of any claim of the parent or any cited documents could beconstrued as a disclaimer of any subject matter supported by the presentdisclosure, Applicants hereby rescind and retract such disclaimer.Accordingly, the references previously presented in the parentapplications may need to be revisited.

What is claimed is:
 1. A computer-implemented method, comprising:receiving, by a computing system, a translation for content from asubmitter of the translation, the content being associated with an itemin an electronic marketplace; providing the translation to a pluralityof reviewers, a reviewer of the plurality of reviewers having a reviewerscore; receiving a rating from the reviewer of the plurality ofreviewers, the received rating indicating an opinion of the reviewerwith respect to the translation; determining, by the computing system, asubmitter score for the submitter of the translation based at least inpart on the reviewer score and the received rating; determining atranslation score for the translation based at least in part on thesubmitter score; and providing the translation for presentation to auser of the electronic marketplace based at least in part on thetranslation score.
 2. The computer-implemented method of claim 1,further comprising ranking the translation score with respect to scoresof other translations of the content.
 3. The computer-implemented methodof claim 2, wherein the translation is provided for display furtherbased at least in part on the ranking of the translation score withrespect to the scores of the other translations of the content.
 4. Thecomputer-implemented method of claim 2, wherein the other translationsare computer-generated translations.
 5. The computer-implemented methodof claim 1, wherein the submitter score indicates a quality rating forthe submitter that is based at least in part on a number of previouslysubmitted translations.
 6. The computer-implemented method of claim 1,wherein the reviewer score indicates a quality rating for the reviewer.7. The computer-implemented method of claim 6, wherein the qualityrating for the reviewer indicates a degree of similarity between ratingssubmitted by the reviewer for particular content and other ratingssubmitted by other reviewers for the particular content.
 8. A system,comprising: at least one computing device configured to implement one ormore services, wherein the one or more services are configured to:obtain a plurality of translations of content from one or moresubmitters; receive one or more ratings of the plurality of translationsfrom one or more reviewers; determine a plurality of respectivesubmitter scores for the one or more submitters based at least in parton the one or more ratings; rank the plurality of translations based atleast in part on the plurality of respective submitter scores; andprovide, for presentation to a user, a particular translation of theplurality of translations based at least in part on the ranking.
 9. Thesystem of claim 8, wherein the content is in a first language andwherein the provided particular translation is in a second language. 10.The system of claim 8, wherein the memory includes further instructionsthat cause the system to determine a reviewer score for an individualreviewer of the one or more reviewers based at least in part on acomparison between a first rating for the content received from theindividual reviewer and a second rating for the content received from adifferent reviewer.
 11. The system of claim 10, wherein the ranking isfurther based at least in part on the reviewer score.
 12. The system ofclaim 8, wherein the memory includes further instructions that cause thesystem to obtain a user language preference of the user.
 13. The systemof claim 12, wherein the particular translation is provided to the userfurther based at least in part on the obtained user language preference.14. The system of claim 8, wherein the memory includes furtherinstructions that cause the system to determine whether the plurality oftranslations has met a threshold number of translations.
 15. The systemof claim 14, wherein the memory includes further instructions that causethe system to request additional translations from the one or moresubmitters based at least in part on a determination that the pluralityof translations has not met the threshold number of translations.
 16. Anon-transitory computer-readable storage medium having stored thereoncomputer-executable instructions that, when executed by a processor,cause the processor to perform operations comprising: obtaining aplurality of translations of content from one or more submitters;receiving one or more ratings of the plurality of translations from oneor more reviewers; determining a plurality of reviewer scores for theone or more reviewers based at least in part on the one or more ratings;determining a plurality of submitter scores for the one or moresubmitters based at least in part on the plurality of reviewer scoresand the one or more ratings; determining a plurality of translationscores for the plurality of translations based at least in part on theplurality of submitter scores; selecting a highest scored translation ofthe plurality of translations scores; and providing, for presentation toa user, the highest scored translation.
 17. The non-transitorycomputer-readable medium of claim 16, having further instructions that,when executed by the processor, cause the processor to performoperations comprising determining a primary language for the user. 18.The non-transitory computer-readable medium of claim 17, wherein theparticular translation is provided to the user further based at least inpart on the determined primary language of the user.
 19. Thenon-transitory computer-readable medium of claim 16, having furtherinstructions that, when executed by the processor, cause the processorto perform operations comprising determining whether an individualtranslation score of the plurality of translation scores has exceeded athreshold value, the individual translation score being associated withan individual translation of the plurality of translations.
 20. Thenon-transitory computer-readable medium of claim 16, having furtherinstructions that, when executed by the processor, cause the processorto perform operations comprising storing the individual translation whenthe individual translation score exceeds the threshold value anddiscarding the individual translation when the individual translationscore does not exceed the threshold value.